The invention relates to the control of complex, general (nonlinear), discrete-time, e.g., physical, socioeconomic or biological, systems in which not only the parameters but the equations governing the system are unknown. More specifically, the invention is a method for estimating a system controller without building or assuming a model for the system. The controller is constructed through the use of a function approximator such as a neural network or polynomial which, in turn, uses a stochastic approximation algorithm. The algorithm is based on a simultaneous perturbation gradient approximation which requires only system measurements, not a system model.
Frequently, a system designer must determine a means to control and regulate a system when there is uncertainty about the nature of the underlying process. Adaptive control procedures have been developed in a variety of areas for such problems (e.g., manufacturing process control, robot arm manipulation, aircraft control, etc.), but these are typically limited by the need to assume that the forms of the system equations are known (and usually linear) while the parameters may be unknown. In complex physical, socioeconomic, or biological systems, however, the forms of the system equations (typically nonlinear), as well as the parameters, are often unknown, making it impossible to determine the control law needed in existing adaptive control procedures.
Numerous current methods use a function approximation (FA) technique to construct the controller for such systems. Associated with any FA will be a set of parameters that must be determined. Popular FAs include, for example, polynomials, multilayered feed-forward neural networks, recurrent neural networks, splines, trigonometric series, radial basis functions, etc.
By results such as the well-known Stone-Weierstrass approximation theorem, it can be shown that many of these FA techniques share the property that they can be used to approximate any continuous function to any degree of accuracy (of course, this is merely an existence result, so experimentation must usually be performed in practice to ensure that the desired level of accuracy with a given FA is being achieved). Each FA technique tends to have advantages and disadvantages, e.g., polynomials have a relatively easy physical interpretability, but the number of parameters to be determined grows rapidly with input dimension or polynomial order.
Others have considered the problem of developing controllers based on FAs when there is minimal information about the system equations. The vast majority of such techniques are indirect control methods in the sense that a second FA is introduced to model the open-loop behavior of the system. This open-loop FA is typically determined from sample input-output "training" data on the system prior to operating the system in closed loop and constructing the controller FA.
It is not possible in this model-free framework to obtain the derivatives necessary to implement standard gradient-based search techniques (such as back-propagation) for estimating the unknown parameters of the FA. Stochastic approximation (SA) algorithms based on approximations to the required gradient will, therefore, be considered. Usually such algorithms rely on well-known finite-difference approximations to the gradient. The finite-difference approach, however, can be very costly in terms of the number of system measurements required, especially in high-dimensional problems such as estimating an FA parameter vector (which may easily have dimension of order 10.sup.2 or 10.sup.3). Further, real-time implementation of finite-difference methods would often suffer since the underlying system dynamics may change during the relatively long period in which measurements are being collected for one gradient approximation.
There is therefore a need for a control procedure that does not require a model for the underlying system.